Topological Singularities in Cortical Orientation Maps: the Sign Theorem Correctly Predicts Orientation Column Patterns in Primate Striate Cortex

Optical imaging methods have revealed the spatial arrangement of orientation columns across striate cortex, usually summarized in terms of two measurements at each cortical location: (i) a \best" stimulus orientation, corresponding to the stimulus orientation that elicits a maximal response , and (ii), the magnitude of the response to the best orientation. This mapping has been described as continuous except at a set of singular points (also termed \vortices" [1] or \pinwheels" [2] ). Although prior work has shown that vortex patterns qualitatively similar to the ones observed in visual area 17 of the Macaque cortex can be produced by either band-pass [3] [4] or low-pass [5] ltering of random vector elds, there has been to date little further topological characterization of the structure of cortical vortex patterns. Nevertheless, much theoretical work has been done in other disciplines on mappings analogous to the cortical orientation map. In particular, a recent theorem in the optics literature termed the sign principle [6] states that adjacent vortices on zero crossings of a phase (orientation) mapping must always alternate in sign. Using digitized samples of recently published optical recording data in monkey striate cortex [7] we show that the cortical orientation data does indeed possess 100% anticorrelation in vortex sign for next-neighbor vortices, as predicted by the sign theorem.This provides strong experimental support for the assumptions of continuity of cortical vortex maps which underly the sign theorem. Similar analysis predicts a lack of \higher order" vortices in the cortical orientation map, which is also found to be in agreement with optical imaging observations. It also follows from this work that cortical vortices must be created simultaneously in clockwise-anti-clockwise pairs. This suggests a possible basis for a modular (hyper-columnar) relationship among pairs of cortical vortices that originate at the same developmental time. In summary, this work indicates that primate visual cortex orientation column structure is best understood in the context of other 2 \ordered continuous media", (e.g. liquid He3, cholesteric liquid crystals, random optical phase maps, to name only a few) in which an order parameter (orientation in this case) is mapped to a physical space, and in which the topological properties of the mapping determine the observable regularities of the system. We also point out that these methods may well be applied to a variety of other cortical map systems which admit an \order parameter", i.e. for which each cortical position is assigned a continuous stimulus value. 3 The population response in cortex to oriented stimuli, as measured in optical imaging experiments, is characterized by the following method (e.g. see [1]). The optical response i(x; y) at each cortical location (x; y) is measured for N di erent stimulus orientations, i (spanning the entire range of possible orientations, [0:: ]). This yields N pairs of values: ( i(x; y); i) (i = 1 : : :N ); (1) where i(x; y) is the magnitude of the measured response at cortical location (x; y) to a stimulus with orientation i. The order parameter space for orientation consists of i R1 mod . Because the operators 'sin' and 'cos' are de ned in terms of a circular domain with 0 = 2 , it is convenient to multiply the orientation i by 2 to yield ̂i = i 2 , whose range is [0::2 ] 1. The N pairs of optical imaging measurements then become ( i(x; y); ̂i) (i = 1 : : :N ): (2) These pairs are then combined into an orientation response function ( (x; y); (x; y)), by (i) considering each pair in Eqn. 2 to represent the complex number i(x; y)êi(x;y); (3) [1] and (ii) summing, at each (x; y), the real and imaginary parts of the complex number (Eqn. 3), separately, for each of the N pairs, fRe(x; y) = N Xi=1 i(x; y) cos ĥi(x; y)i (4) fIm(x; y) = N Xi=1 i(x; y) sin ĥi(x; y)i ; (the result of this stage of processing for optical imaging data from the macaque striate cortex is shown in Fig. 1) and (iii) converting the real/imaginary representation of Eqn. 4 back to a polar representation and returning to the range [0:: ] using the transformation ( (x; y); (x; y)) = pfRe(x; y)2 + fIm(x; y)2; 1 2 arctan fIm(x; y) fRe(x; y) : (5) 1Both orientation and direction mappings are topologically equivalent and the arguments in this work thus apply equally to both. This is because the unit circle S1, corresponding to the range [0;2 ] is topologically equivalent to P 1 (one dimensional projective space) corresponding to the range [0; ]. This equivalence follows immediately from the existence of a homeomorphism connecting them, namely multiplication by 2. 4 a b fRe(x; y) fIm(x; y) FIG. 1. Complex variable representations of optical imaging data frommacaque striate cortex, obtained by processing raw data in the form of Eqn. 1, using Eqn. 4. The raw data are obtained by digitally scanning eight frames each corresponding to a di erent orientation, i in Eqn. 1 ( 1 = 0 ; 2 = 45 ; 3 = 90 ; 4 = 135 ; 5 = 180 ; 6 = 225 ; 7 = 270 ; 8 = 315 ), from Fig. 5 of [7] (used with the author's permission). The frames shown are 3 4mm in size; brighter gray levels indicate positive ranges for fRe and fIm while black regions indicate negative ranges. A polar representation of this data is shown in Fig. 2, and the zero crossings of fRe and fIm are shown in Fig. 4. The orientation response function of Eqn. 5 has been used to represent the output of optical recordings of orientation selectivity in striate cortex (e.g., see [7]), in which (x; y) is interpreted as the stimulus orientation that best elicits a response at cortical location (x; y) and (x; y) corresponds to the absolute magnitude of that response. The cortical position variables (x; y) will henceforth be dropped for brevity. The reconstructed ( ; )representation (Eqn. 5) of optical imagingdata from the macaque striate cortex [7] 2 is shown in Fig. 2. Optical imaging methods have indicated a dramatic pattern of orientation singularities to exist in primate and cat V-1 [1, 2] (see Fig. 2). The terms \vortices" [8, 4], \pinwheels" [2], \dislocations" [9], \singularities", \orientation singularities" and \phase singularities" [6] have been used interchangeably to refer to singular points of an orientation mapping, around which the entire range of possible orientations [0:: ] is continuously represented. All possible orientations are represented in the vicinity of a singularity, allowing for easy visualization of singularities in gray-scale coded orientation maps 2We thank Dr. Gary Blasdel for generously making the data used in this paper available to us in digital form. 5 a (x; y) b (x; y) FIG. 2. ( ; ) representation of optical imaging data [7], obtained by using the transformation to polar coordinates (Eqn. 5) of the data shown in Fig. 1. (a) The cortical orientation response, showing \best" orientation response at each cortical location. Neighboring gray values correspond to neighboring orientation, with white = black = 0 . Gray-scale values are binned into 8 bins (this is done only for visualization { it is easier to visualize the vortex centers by eye in this banded representation { the binning is never employed in any processing stage). (b) The corresponding amplitude map of the orientation response, showing at each cortical location the absolute magnitude of the response to the corresponding best orientation in (a). An automatic vortex detection algorithm was employed on the raw data (Eqn. 2) to nd the vortex centers and their signs, which are shown in both (a) and (b) by the superimposed markers denoting vortex signs and locations. Positive (counterclockwise) vortices are shown as circles, while negative (clockwise) vortices are shown as squares. One can verify the properties of singularities in both frames by noticing that (i) in panel (a) ( (x; y)) all orientations surround each vortex center; (ii) vortices appear at the local minima of frame (b) ( (x;y)). 6 (Fig. 2a). The characteristic feature of the V-1 orientation response function (Eqn. 5) is the existence of local spatial correlation in the orientation response. This spatial correlation of orientation was originally termed \sequence regularity" [10]: nearby neurons or locations in cortex are characterized by nearby values of best orientation response. The modi cation to this statement, as a result of optical imaging, has been the observation of singular points around which all orientations are locally represented. There are a wide range of analogies in the physics of continuous media to the striate cortical vortex pattern shown in Fig. 2 (see [11] for a review): vortex patterns in liquid Helium3, cholesteric liquid crystals and random optical phase elds [12], to name just a few analogies 3. All of these systems, as well as striate cortex, are best understood in terms of the topological consequences of assigning an \order parameter" (in this case, the orientation response in cortex, represented as an angle) to each point of some underlying physical space (in this case, imaged \pixels" of the twodimensional cortical sheet). The order parameter describing orientation, namely angle, may be viewed in topological terms as the boundary of the unit circle S1, in the sense that points in S1 are in one-to-one correspondence with possible orientations. S1 is not simply connected 4. The cortical plane, however, is topologically modeled as a subset of R2, the two-dimensional real number manifold, which is simply connected. Cortical orientation maps therefore are instances of the mapping S1 ! R2, between spaces of di erent connectivity (i.e., di erent homotopy classes [11]). A continuous mapping between spaces of di erent connectivity must contain singularities such as the vortices observed in cortex [11], as has been previously discussed [5] and recently observed in optical recording experiments (Fig. 2). In prior work, it has been demonstrated that random orientation maps, when spatially ltered either by a bandpass [15, 3] or a low-pass [5] lter, provide similar vortex maps to those observed in optical recording. These methods of generating vortex maps all rely on the lter's introduction of local correlation (and thus, \continuity") in the mapping and on the fact that the mapping is between spaces of di erent connectivity, to produce vortex patterns. These same spatial lter methods have been applied more recently for constructing random vortex models of cortical orientation [16, 17]. 3The relationship of the cortical orientation pattern to vortex patterns in uid mechanics was rst noted in [8], and the analogy to patterns of phase disorder, as in cholesteric liquid crystals was discussed in [13]. The suggestion that cortical orientation maps are a phase singularity pattern has also been previously suggested [14]. The relationship of cortical orientation to ltered patterns of spatial orientation noise was rst discussed in [3]. 4A simply connected space is one in which all closed curves may be smoothly shrunk to a point. 7 Mathematically, a vortex is characterized by two key properties of the orientation mapping (x; y) and the amplitude mapping (x; y). First, all orientations are represented, at least once, around the singularity center. Equivalently, the contour integral along any loop in (x; y) that surrounds a (single) vortex is always an integral number of 2 radians. This follows from the presumed continuity and single-valuedness of the cortical mapping everywhere except at the singularity itself. Since the mapping is continuous, when traversing a loop surrounding a vortex from starting point (x0; y0) eventually returning to (x0; y0), one must also return along the corresponding path in orientation space to the angle 0 that was mapped onto (x0; y0). The return to the same angle can only happen in two ways, either (a) the entire orientation mapping is (uninterestingly) constant or (b) as the loop in cortex is traversed, the underlying orientations encountered also form a loop in orientation space of 0,1,2,...n \windings". It turns out that a singularity must occur inside the loop when case (b) above is satis ed for n 1 [11]. In this case, the sum of angle di erences along a loop surrounding a singularity, as it is traversed in a clockwise direction, is an integer multiple of 2 radians. This integer, which can be positive or negative (corresponding to a positive or negative increase in angle along a clockwise traversal of the loop, respectively) is termed the winding number. In summary, a singularity exists if and only if its winding number is nonzero. The sign of a vortex is de ned as the sign of its winding number 5. In the amplitude mapping, (x; y), a singularity is characterized by the amplitude of the response function approaching zero in the vicinity of the vortex center and equaling zero at the center. The amplitude mapping at singularities in a discrete medium (such as a video image obtained by the optical imaging method) may di er slightly from zero due to discretization (both the underlying neuronal system and its digital optical image are discrete). Nevertheless, singularities in discrete mappings always occur at local minima of the mapping, in our experience, as expected (and explained) by the continuum arguments summarized above, and in the literature of ordered media [11]. The main result of the present paper is that a theorem developed in the context of random optical wave elds [6] termed the sign principle accurately characterizes the primate V-1 orientation map. The sign principle states that adjacent singularities must always alternate in sign along any zero crossing 5See also [7], which discusses the sign of singularities with reference to Ridge systems [18] { in particular, why the \+" and \-" singularities must be present in equal numbers. The implications of this work to vortex pairwise creation and annihilation are further discussed below. 8 path of the real (fRe) or the imaginary (fIm) representation of an orientation map. Fig. 3 illustrates the proof of the sign principle (based on the proof in [6]). Fig. 4 demonstrates that the sign-principle correctly predicts the structure of primate V-1 orientation maps measured by optical imaging [7] 6. This paper places studies of cortical neuroanatomy into the well developed context of continuous media, which is quite distinct from the \neural network" point of view where the discrete nature of neural structure is emphasized. This \supra-neuronal" [19] level of description of cortical architecture is supported by the validity of the sign principle in accurately characterizing the cortical orientation map. Speci cally, a wide range of phenomena related to cortical orientation map structure is clari ed by the use of the sign principle. These will now be brie y outlined. The real and imaginary zero crossings of the observed orientation map provide a natural coordinate system for analyzing the V-1 orientation map. For example, the pattern statistics of the vortex map (e.g. the probability of neighboring vortices being of opposite sign) appears to be a random variable when analyzed in spatial (i.e. conventional Euclidean) metric but appears to be rigorously correct when the metric is computed along zero crossing lines, as per the sign principle. Further statistical characterization of cortical orientation structure is likely best done by taking the zero-crossings coordinate system into account. Nevertheless, even when one looks at vortex signs using the conventional Eucledian metric, a high degree of nearest neighbor correlation is observed in natural vortex patterns. Statistical studies using the Eucledian metric have shown 85-90% anti-correlation in sign among vortices in optical interferometry experiments [6] and later, the same anti-correlation percentages were computed for vortices in optical imaging data [20]. The following heuristic statistical argument based on the sign principle, predicts that on average, 87.5% of all nearest-neighbor vortex pairs are anticorrelated in sign [6]: 6In order to produce Fig. 4, and for further analysis that takes into consideration vortex locations, we have developed and empirically evaluated an accurate vortex detection algorithm. The main steps of the algorithm consist of (i) nding all pixels in the image whose amplitude is smaller than the amplitude of all 8 nearest neighbor pixels; (ii) integrating the angle-di erences along a traversal of the 8 nearest neighbor pixels and along a slightly larger circle (radius = 2 pixels) centered on each pixel, in order to compute the winding number (and thus also the vortex sign) and ensure that the winding number is close to an integral multiple of 2 radians. All vortices in Figs. 1, 2 and 4 were automatically detected using this algorithm, which produced no errors (as veri ed by visual checking) for the orientation map used in this analysis and for other experimental and computer generated orientation maps. 9 a b (x; y) (x; y) c d [fRe(x; y)]+ [fIm(x; y)]+ e f x y ZRe FIG. 3. Proof of the sign principle. The sign principle states that adjacent vortices on any zero crossing path ZRe (ZIm) of the real (imaginary) representation, fRe (fIm), of Eqn. 4 must alternate in sign, where the vortex \sign" is positive if the direction of increase in orientations around the vortex is counterclockwise, and negative otherwise. (a) and (b) are a small section of optical imaging data taken from Fig. 2. (a) Cortical orientation preference denoting the orientation that elicits a best response at each cortical location. (b) Magnitude of the orientation response in (a), showing expected local minima at vortex centers. (c-d) Real/imaginary representation of the mapping in (a,b), obtained according to Eqn. 4, and thresholded at zero, positive regions are black. (e) The zero crossings of the mappings in (c) and (d), superimposed. Gray lines: the zero crossing paths of fRe; black lines: the zero crossing paths of fIm; horizontalline shading: regions in which fRe > 0; vertical-line shading: regions where fIm > 0; S1 and S2 denote the locations of two singularities in this mapping. (f) A schematic of the mapping near the singularity S1 in (e) showing the coordinate system used at each singularity in the sign principle proof below. To prove the sign principle we imagine walking along ZRe from S1 to S2. At each point (xi; yi) on ZRe we use the coordinate system whose origin is (xi; yi), y-axis is parallel to the tangent of ZRe at (xi; yi), x-axis is orthogonal to the y-axis and the positive part of the x-axis is aligned with the positive part of fRe, as shown in (f). It can be shown [9] that the sign of a vortex is given by the sign of @fRe @x @fRe @y @fIm @x @fIm @y = @fRe @x @fIm @y @fRe @y @fIm @x . In the chosen coordinate system, @fRe @y is always zero because the y-axis is always tangent to ZRe. Moreover, as we move along ZRe from S1 to S2, @fRe @x is always positive since (by our initial assumption) fRe always increases to our right and decreases to our left. Thus, the sign of a singularity encountered along our path is solely dependent on the sign of @fIm @y . At S1, @fIm @y < 0 (because fIm decreases in the direction pointing from S1 to S2, opposite to the increasing direction of the y-axis), implying that S1 is a negative vortex. Since fIm is negative between S1 and S2 and since fIm must increase back to zero at S2 (because singularities occur only at the intersections of zero crossings), at S2 @fIm @y > 0 and thus S2 must have an opposite sign to S1. By symmetry, the same proof applies to traversal of ZIm and for the other possible directions of increase of fRe and fIm at S1. 10 FIG. 4. Demonstration that the sign principle holds for a previously published cortical orientation map, reconstructed from the data [7] shown in Figs. 1 and 2. Gray lines denote the zero crossings ZRe of fRe in Fig. 1 while black lines correspond to ZIm. Vortices occur only at the intersections of zero crossings ZRe and ZIm. The vortex locations and signs are shown by either circles or squares, corresponding to positive or negative signs, respectively. Adjacent vortices on any zero crossing path always alternate in sign, as predicted by the sign principle. Assume: if two vortices are nearest neighbors then they are adjacent on some zero-crossing path. The sign principle implies that if two vortices are adjacent on some zero-crossing path, then they must have opposite signs. Pick any two adjacent vortices, these may share: (1) ZRe, (2) ZIm, (3) ZRe and ZIm, or (4) No zero-crossing paths in common. Assuming that each of the four above possibilities are equally likely, 3/4=75% of the vortices are adjacent on some zero crossing, therefore 75% of the nearest neighbor vortex pairs are anticorrelated in sign. Assume remaining 25% have randomly paired signs, then 12.5% of them will be anti-correlated in sign. Total: 75+12.5=87.5% of vortices are anti-correlated. The sign principle implies, therefore, that in the Eucledian domain the signs of neighboring vortices ought to be highly (87.5%) anti correlated. The predicted and observed anti-correlation stems from the fact that most near neighbor vortices share a common zero crossing, which implies that the vortex pair has opposite signs. Computer simulations of low-pass and bandpass ltered (complex) white noise 11 patterns produced 87% 4% and 89% 4%, respectively [20]. The simulations of [6] had 90% 2%. Vortex nearest-neighbor pairs (de ned as vortex pairs in which one vortex center is the nearest neighbor to the other vortex center, which on average account for approximately 60% of all possible vortex pairs [6]) are, as expected, more highly anti-correlated in sign (94% 3%) in both bandpass and low-pass ltered complex noise. As seen in this paper, winding number characterizes the local structure of cortical \vortices". \Winding number" has been termed \topological charge", since it obeys a form of conservation law which is similar to that of conventional (electro-magnetic) charge conservation [21]. Brie y, vortices are created and destroyed in \pairs", as we have frequently observed in computer simulation. The sign principle shows why this is the case: vortices are created when a real [resp. imaginary] zero crossing curve \wanders" across an imaginary [resp. real] one. Generically, two vortices (of opposite sign) will be produced simultaneously by this mechanism, since the zero crossing lines are generically closed and therefore intersections come in pairs 7. The same argument in reverse describes vortex \annihilation", caused for example by smoothing or averaging. Thus, if one low-pass lters a random orientation map, the low-pass lter causes nearby positive and negative vortices to be \annihilated", i.e. to disappear, leaving a singularity free region, as shown in [5]. The sign principle thus provides deep insight into the appearance or disappearance of vortex pairs. Moreover, as has been noted before [5], any form of spatial ltering (either low pass or band-pass) is capable of producing new vortices or annihilating existing vortices. Therefore, care must be observed in digital manipulation of optical recording data, especially because (somewhat counter-intuitively) a \blurring" operation such as low pass ltering can actually produce extremely \sharp", yet artifactual, vortex structure. Winding numbers greater than 1 have not been observed in V-1 optical recordings. This can be understood with reference to the pair-creation mechanism, occurring when two zero crossing paths have intersected. Higher order vortices require more than two zero crossing lines to intersect at a point. Aside from being statistically unlikely, this situation is also topologically unstable, since it degenerates under in nitesimal perturbation into distinct rst order vortex pairs. 7This statement can be violated by a situation of tangency between the real and imaginary zero crossing, but this situation is topologically unstable, i.e. collapses into the pair-creation mechanism under a small perturbation. By the same argument, a situation where the real and imaginary zero crossings intersect at a line (yielding a line-singularity) rather than at a point is also topologically unstable and therefore extremely unlikely; moreover, a line singularity would make cortex non simply connected, contrary to our assumptions. 12 Vortices are created in pairs of positive-negative winding number. This provides a natural iden-ti cation of neighboring cortical singularities, identifying those pairs of adjacent vortices (in the zerocrossing metric) which were created at a single zero crossing intersection event, i.e. adjacent singulari-ties along a zero crossing path that were created simultaneously in a vortex pair creation. In practice,there appears to be no way to create this labeling after the fact, but it could be done in principleby following developmental trajectories. Moreover, this identi cation of cortical vortex pairs seems toprovide a solution to the long existing question of whether there is a modular supra-columnar (i.e.hypercolumn like) pattern in V-1, as originally suggested by Hubel and Wiesel [10]. Since corticalvortex pairs must be created simultaneously, via intersection of two zero crossing contours, there is anatural pair-wise identi cation of nearby vortices which can be interpreted as providing a basis for a\hypercolumn" like pairing of singularities.Finally, it is important to note that the entire analysis provided here, in terms of the topologicalproperties of ordered media, is likely to carry over to other cortical feature maps. For example, thedirection column structure of MT cortex [22] is also a map of S1 ! R2, and thus would be expectedto provide similar supra-neuronal behavior as outlined above for the V-1 orientation column map.Further, cortical feature maps can be described in terms of the same language used here, i.e. theassignment of an order parameter (i.e. the feature space) to a physical space (i.e. the cortex). Thetopological properties of these two spaces are then expected to determine the qualitative structure ofarbitrary cortical feature maps in the manner which has already been well characterized in the contextof the topological defects of ordered media [11]. Thus, supra-neuronal architectures in cortex may be,in addition to being accessible to experimental techniques such as optical recording, accessible to a welldeveloped body of theoretical methodology.ACKNOWLEDGMENTS. We Thank Dr. Gary Blasdel for his contribution of raw optical imag-ing data.CORRESPONDENCE should be sent to E.L. Schwartz (email: [email protected]). This worksupported by grants NIMH 5R01MH45969 and ONR N00014-95-1-0409.13 References[1] Blasdel, G. and Salama, G. Nature 321, 579{585 (1986).[2] Bonhoe er, T. and Grinvald, A. Nature 353, 429{431 (1991).[3] Rojer, A. and Schwartz, E. L. Biological Cybernetics 62, 381{391 (1990).[4] Blasdel, G. Abstracts, Society for Neuroscience 17, 1088 (1991).[5] Schwartz, E. and Rojer, A. In ICPR Proceedings, ICPR-12. International Conference on PatternRecognition, (1994).[6] Shvartsman, N. and Freund, I. J. Opt. Soc. Am. A 111(10), 2710{2718 (1994).[7] Blasdel, G. Journal of Neuroscience 12(8), 3139{3161August (1992).[8] Schwartz, E. L. Biological Cybernetics 28, 1{24 (1977).[9] Nye, J. F. and Berry, M. V. Proc. R. Soc. Lond. A 336, 165{190 (1974).[10] Hubel, D. H. and Wiesel, T. N. J. Comp. Neurol. 158, 267{293 (1974).[11] Mermin, N. D. Rev. Modern Physics 51, 591{648 (1979).[12] Shvartsman, N. and Freund, I. Physical Review Letters 72(7), 1008{1011 (1994).[13] Schwartz, E. L. Biological Cybernetics 37, 63{76 (1980).[14] Winfree, A. T. When Time Breaks Down. Princeton University Press, Princeton N.J., (1987).[15] Rojer, A. and Schwartz, E. L. IJCNN Proceedings II, 603{613 (1989).[16] Blasdel, G. and Obermayer, K. Neural Networks 7, 865{881 (1994).[17] Grossberg, S. and Olson, S. J. Neural Networks 7, 883{894 (1994).[18] Penrose, R. Ann. Hum. Genet. 42, 435{444 (1979).[19] Schwartz, E. L., editor. Computational Neuroscience. MIT Press, Cambridge, MA, (1990).14 [20] Tal, D. and Schwartz, E. L. Invest. Opthal. and Visual Sci.. Supplement 37, 938 (1996).[21] Finkelstein, D. Annals of Physics 6, 230{243 (1959).[22] Albright, T. D. J. Neurophysiology 52, 1106{30 (1984).15